Stephen Downes
Knowledge, Learning, CommunityThere's not a whole lot of time to get into this in great depth, but I do want to introduce this screen here, which is the game no man's sky, which is available in 3D. I've spent thousands of hours in this game. I probably shouldn't but I love it.
So what is the metaverse? There's a few things that come together to create the metaverse: extended reality, virtual reality, all of that. I'm going to talk a bit about what extended reality and virtual reality are, and talk about the different opportunities for its use in education. And then I'll wrap up with some of the issues and concerns about the technology.
So we begin with virtual reality. Virtual reality is a case where users enter a completely virtual world. They use a mask or a headset so that they're completely immersed in this environment. All sights, sounds and objects are simulated by the computer
Compared to that we have augmented reality. This is where you're seeing the real world through a see-through viewing device like. Tthere used to be Google Glass. There's Hololens, and there will be others. And then alongside the real world, you see data projected, sort of like a heads up display (HUD). And what this does is it offers more information, typically about what you are seeing.
As well, we have mixed reality. This is blending or merging of the virtual and real environment, so you're working probably with a headset. You're manipulating something in virtual reality, but the effect or impact of what you're doing is felt in the real world. So for example, on the right there, you see a guy manipulating a machine and out there in the world is the real-life machine. We also see this used for for example, for flying drones or things like that.
Extended reality is a term you'll hear a lot, and what it refers to is VR AR and MR collectively.
And you might be wondering, then what is the metaverse. The metaverse combines all of these XR technologies. And then it adds to it persistent digital objects. There are different kinds of objects that can be added to the real or virtual world, for example: digital currencies such as Bitcoin, or Ethereum; digital identities, or DIDs; or things like non-fungable tokens programs; software, etc. There's a whole range of persistent digital objects.
So how do we use this in education and learning?
One major use is visualization. Small things can be magnified, or we can look inside things. Here we see, for example, somebody looking at the inside of a brain through a headset. That's a computer vision and graphics project that we're running here at NRC in Canada. You can also speed up or slow down time.
Another thing you can do is interactivity. Interactivity allows for the hands on manipulation of objects. Here, we have an example from the United States where they are manipulating virtual objects, in this case dice, and they're using a virtual hand. In order to do that, we, we can see how this could be used to learn how to control machines or devices. It's also useful for the illustration of mechanical or physical principles. I've seen, for example, you can control the oscillation of digital music by controlling virtual controls.
Another major use is to create safe environments to learn about things that are inherently dangerous flying a plane. For example, fighting a fire working with high voltage equipment. This is a project here - and the the paper is still forthcoming but has been accepted - that we're doing at NRC. This is a project that I've personally worked on with the team here at NRC for emergency response training for first responders in cases of accidents or spills involving hazardous materials like gas or methane or ammonia or things like that. And there's a whole range of things that people have to do. So we have a virtual emergency response guidebook for example virtual binoculars and then the scene assessment you do etc.
Another way that VR helps is with accessibility and there's two ways that this plays out. First of all it helps people with limited mobility or who live in remote regions or who have time constraints, where hands on equipment is not available. So it's an excellent tool to assist with accessibility. By the same token, and we've seen this quite a bit, it helps people learn how to support people with diverse needs and and so you're able to learn for example how to work with a mentally challenged person or a physically challenged person to build empathy or to improve cognitive ability. And there's an NRC project here involving that, without actually having to practice on people who have mental or cognitive challenges. And so this makes the the the training something that is much more widely accessible.
VR, AR, and the rest also support remote collaboration. This is a really exciting area because it allows people to create or manipulate common digital objects, and this is where this whole concept of object persistence comes into play, because you are able to know that you and the other person are creating or manipulating the same objects. In games like no man's sky we call this multiplayer mode, where you have multiple players interacting with the same world. In the case of education and training (we don't call it multiplayer. maybe multi-user mode) we use it for practical educational applications. Illustrated here is a digital circuit design class in Poland where multiple people come in, they're looking at digital circuit design and of course it's greatly expanded so they can see it. And then together their manipulating the circuits in order to create integrated circuits.
Also useful for education and training is the capability of recording a performance in a virtual environment. These are really useful. First of all, they can be viewed later as an object for discussion. Or the person who did the action can debrief, or go over what they did. And it's not like just taking a video of a physically real performance. In virtual reality, we can see the performance from all angles and all directions, we can zoom in on subtle things. And then as well these recordings can be used as learning resources and so people can access the resource and go into it after just to see what was done, say, by an expert in the field.
Where the really exciting things happen with the metaverse is, when the XR technology and persistent objects are combined with other technologies. This is where the real opportunities will happen. So, I'll look at a few of these.
First of all, combining it with artificial intelligence. This adds a whole new dimension to virtual reality. First of all you can get life-like immersive environments. I mentioned No Man's Sky. We actually interact with an environment that is AI generated. So we have to have negotiations with creatures, for example, or animals that behave like real animals. In education, we can have smart agents that interact with users that generate tasks or resources or that make inations based on user activities. As well, artificial intelligence can be used to evaluate learning in the XR environment. Here, we have an example of AI enabled VR tuition (that is, instruction) in China.
Next, combined with cryptography, In other words, with things like digital signatures or blockchain networks we can create distributed and multi-user virtual environments. So in other words, you don't all go to the same virtual place. You're each in different places looking at a common reality. This supports massive simulations, sporting events, financial markets. It's the something that the military is very interested in. Or we can look at complex global phenomena, like climate change. As you can see, we move from just one environment to multiple environments, where connecting these environments are the individuals and the persistent digital objects.
We can also combine VR and AR with real-time data based on real-world models and statistics, For example, imagine a football game or a hockey game is being played in real time. We can actually enjoy watching that game from the perspective of the player, or from the perspective of on the ice, or on the pitch. Or for learning, we can watch heart surgery from the perspective of the surgeon or the heart in real time as they are actually performing the operation.
Combining it with haptics. Haptics are feedback devices that give you a sense of touch or feel of vibrating controller, for example. Haptics and these forms of sensory feedback can be used with VR to help you learn skills, where the skill is a mechanical or physical skill. Here we have an NRC project called NerroVR which trains people how to do brain surgery, and specifically, removal of tumors with medical instruments. And you actually feel the resistance of the tumor as you try to eliminate it.
Now, there are, of course, risks and potential issues with this technology.
First of all, cost is a major factor. You know, a low cost headset may cost a thousand dollars Canadian. It'll also takes a lot of time to learn how to use this technology because you need to learn how to manipulate objects in the VR world. It's not something that we grow up learning how to do. Maybe our children will know.
There are also health risks. Ergonomic issues are probably the major ones. Repetitive stress injuries. Perhaps nausea or forms of cyber sickness caused by latency issues: that's where you move something, and it takes a second for you to see the results on the screen. Or actual physical injury caused by falling or stumbling using the headset.
There are also concerns about bad actors and malpractice. You know, we can present physical phenomena however we like in the virtual world, and this can lead to people being taught falsehoods about the real world. There are also cases of inappropriate use of surveillance technologies, and and even cases of using virtual world technologies to do things like watch students as they take exams and things like that.
And finally, there are social and cultural issues to be considered VR is used to help people desensitize themselves to things like fears of spider. But this could also lead to desensitizing other elements, and there may be cultural or ethical values that change through virtual world interactions.
So that's what I have to present today, and I thank you for your time and I hope you enjoy the rest of the conference.
Stephen Downes, Casselman, Canada
stephen@downes.ca
Last Updated: Apr 29, 2024 9:32 p.m.
.